1. Technical Field
The invention is related to video tours, and more particularly to a system and process for providing an interactive video tour of a tour site using an image-based rendering technique that enables users to explore remote real world locations.
2. Background Art
Interactive scene walkthroughs have long been an important application area of computer graphics, as evidenced by the efficient rendering algorithms that have been developed for visualizing large architectural databases [9,1]. In fact, many beautiful examples of 3D reconstruction from multiple images, video, and/or range data have been demonstrated by research groups around the world. The most ambitious of these is Seth Teller's City Scanning Project at MIT [2], in which dozens of buildings have been reconstructed from high-resolution, high dynamic range omnidirectional panoramas. While these systems allow the viewer to have unrestricted motion throughout the digitized scene, they fail to capture the visual effects such as reflections in windows, specularities, and finely detailed geometry such as foliage, that make the scene look real.
More recently, researchers have developed techniques to construct photorealistic 3D architectural models from the real world [5,6] and also produced real world tours based on panoramic images [4] and video [3,8]. One such proposed interactive video tour system is described in Lippman's seminal Movie Maps project [7] which collected video clips of driving through the streets of Aspen, and allowed the viewer to navigate through these clips, which were stored on an optical videodisc. Even more recently, many people have demonstrated interactive video walkthroughs using various kinds of omni-directional cameras. For example, interactive video tours based on catadioptric and multi-camera sensors have also been demonstrated [3,8].
What all of these systems have in common is a desire to create a real sense of being there, i.e., a sense of virtual presence that lets users experience a space or environment in an exploratory, interactive manner. This coincides with people's long held interest in travel, in being able to experience interesting and beautiful remote places and to document them with paintings, photographs, and video. However, to date none have been able to provide a viewer with continuous control over both position and viewpoint, as would be desirable to achieve the goal of creating a sense of being there. For example, viewers of Movie Maps had to branch between a fixed number of video clips. In addition, these systems cannot achieve a fully interactive, photorealistic, video based tour on a personal computer at or above broadcast video resolutions and frame rates. The present interactive video tour system overcomes these shortcomings as will now be described.
It is noted that in the preceding paragraphs, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. Multiple references will be identified by a pair of brackets containing more than one designator, for example, [2,3]. A listing of references including the publications corresponding to each designator can be found at the end of the Detailed Description section.